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Abstract — In this paper we investigate stopping criteria for 
iterative decoding from a mutual information perspective. We 
introduce new iteration stopping rules based on an approximation 
of tlie mutual information between encoded bits and decoder 
soft output. Tlie first type stopping rule sets a threshold value 
directly on the approximated mutual information for terminating 
decoding. The threshold can be adjusted according to the 
expected bit error rate. The second one adopts a strategy similar 
to that of the well known cross-entropy stopping rule by applying 
a fixed threshold on the ratio of a simple metric obtained after 
each iteration over that of the first iteration. Compared with 
several well known stopping rules, the new methods achieve 
higher efficiency. 

Index Terms — Iterative decoding, iteration stopping rule, mu- 
tual information 

I. Introduction 

Capacity approaching error correction coding schemes such 
as Turbo codes and LDPC codes are widely adopted in 
wireless standards, e.g.. Turbo codes in 3GPP High Speed 
Packet Access (HSPA) and Long-Term Evolution (LTE) [1] 
IS, LDPC codes in WiMax and Wi-Fi H. Iterative 
decoding is the practical solution for decoder implementation 
in modem chipsets. In order to achieve longer battery life 
and higher throughput, it is necessary to minimize chipset 
power consumption and processing delay. Iteration stopping 
rules serve such purposes by reducing the number of decoding 
iterations while maintaining the performance. In practice, 
cyclic redundancy check (CRC) is often employed for error 
detection which provides an easy solution for early stopping. 
However, not all systems have CRC at physical layer. For 
example, HSPA provides a 24-bit CRC for each transport block 
but no CRC for each code block within a transport block. A 
stopping criterion avoids decoding with the maximum number 
of iterations for every code block. Furthermore, while long 
CRC (e.g., 32-bit CRC) requires higher overhead, short CRC 
(e.g., 8-bit) results in weaker error detection. Iteration stopping 
without CRC is therefore of practical interest. In the following, 
we consider Turbo decoding, but the methodology applies as 
well to LDPC decoding 15]. 

Iteration stopping has been studied since early days of 
Turbo codes. The well known cross-entropy (CE) stopping rule 
||6l uses the relative information between the two constituent 
decoders' soft output as the criterion. Decoding is considered 



as converged and stopped when this relative information is 
close to zero. Based on the same concept, two simplified 
variants of the CE rule were introduced in [7|: The first 
one, sign change ratio (SCR) rule, counts the number of sign 
changes in the extrinsic log likelihood ratios (LLRs) between 
two consecutive iterations. Decoding is terminated when the 
ratio is small enough. The second one, hard-decision-aided 
(HDA) rule, compares second decoder output hard decisions 
with those by the pervious iteration. Decoding stops if all hard 
decisions remain the same. The overall performance by the 
simplified variants are close to those obtained by the origi- 
nal CE rule. Further variants or improvements include: sign 
difference ratio (SDR) [S] extends from SCR by comparing 
sign changes of each component decoder's a priori LLRs and 
extrinsic LLRs; improved hard-decision-aided (IHDA) rule f9| 
modifies HDA to compare hard decisions of two component 
decoders. Aside from savings in memory, the latter two 
variants perform soft/hard decision comparisons after every 
half iteration while previous methods perform comparisons 
after each complete iteration. While other stopping rules are 
also found in the literature, in e.g., fTOl and references therein, 
it is interesting to notice that, above CE, SCR, SDR, HDA, 
and IHDA stopping rules all originate from the cross-entropy 
or relative information perspective of decoding convergence. 

In this paper, we consider decoding convergence estimation 
from a mutual information perspective. In fact, mutual infor- 
mation analysis has been exemplified by the popular extrinsic 
information transfer (EXIT) chart [11 1. In EXIT analysis, the 
mutual information between LLRs and the transmitted bit 
is calculated for system performance evaluation. While it is 
designed as an off-line tool, a similar and simplified process 
may be considered for online convergence estimation. In this 
paper, we show that instead of using true transmitted bits, 
a simple approximation of the mutual information by using 
decoder's hard decision can be used for efficient iteration stop. 

The paper is organized as follows. The system model is 
described in Section along with the approximated calcula- 
tion of mutual information between encoded bit and LLR. The 
first iteration stopping rule is formulated in Section HIl] The 
second stopping rule is presented in Section |IV] Simulations 
are shown in sub-sections lIII-Bl and llV-Al Finally, we conclude 
in Section [Vl 



II. System Model and An approximate mutual 

INFORMATION CALCULATION 

We consider parallel concatenated convolutional code 
(PCCC) |12| where coded bits are mapped to symbols from a 
signal constellation and transmitted over a memoryless channel 
with additive white Gaussian noise (AWGN). At the receiver, 
assume perfect channel knowledge and optimal demodulation, 
iterative Log-MAP decoding 1 13 1 |6|is performed. The mutual 
information between each information bit u G {1, — 1} and its 
associated LLR A can be derived as ifTTl 

/=l-i?A|MJlog2[l + exp(-A)]}. (1) 

where A denotes LLR in general, from which we may further 
specify a priori LLR, Aq, extrinsic LLR, Ag, and a posteriori 
LLR Kapp by adding appropriate subscripts. 

By applying the ergodicity assumption on LLR distribution, 
dill is simplified as [141 

1 ^ 

/ « 1 - - ^ log2 [1 + exp(-u„A(n))] (2) 

where A(n) refers to the a priori LLR for bit u„, n = 
1,2,.-- ,Af. 

Convergence analysis based on above mutual information 
evolution has been utilized by the well-known EXIT chart Bill 
lfT4l . It takes the mutual information between the information 
bit and the associated a priori LLR, la, as the input, and 
generates the mutual information between the information bit 
and its extrinsic LLR, /g, as the output. An EXIT chart is 
generated by plotting the output If, values corresponding to 
a sequence of input la 6 [0,1]. A high mutual information 
indicates high reliabilities of LLRs, and vice versa. 

Note that (|2]i requires knowledge of the information bits 
and therefore is used for off-line analysis. A blind mutual 
information calculation is presented in |15|. However, it re- 
quires more computations. For simple online analysis, we use 
the decoder hard decisions as estimates of the information 
bits, which results in an approximation of the desired mutual 
information /. Clearly, its accuracy depends on the reliability 
of the hard decisions. However, since output LLRs provides 
probabilistic measure about information bits, this approxima- 
tion can approach the maximum value only if LLRs are large 
enough, which in turn indicates the decoding convergence 
lITB l. Therefore, for the purpose of iteration stopping, a high 
threshold value on the approximated mutual information can 
be effective. On the other hand, we are interested in knowing 
the reliability of the overall decoder output, rather than the 
extrinsic information alone. Therefore we may consider the 
mutual information generated by a posteriori LLR, Kapp- In 
fact, using Kapp instead of Ag allows for earlier identification 
of decoding convergence. 

Applying the hard decision by u = sign{Kapp), we approx- 
imate dU of a posteriori LLR, lapp, by 



^app 

1 ^ 

~ ^ ~ log2(l +exp(-Aapp(r;.) • sign{Kapp{n))))) 

1 ^ 

N 

«l_M!^ln(l + e-Kp.(»)l) 

n=l 

= 1 - e 

(3) 

N 

where we define e = ^ ln(l + e^l'^-ff'")!). 

n=l 

Furthermore, adopting the common practice in the log- 
MAP algorihtm implementations |l6l, the calculation of ln(l + 
exp(— can easily handled by a look up table function, 
LUT{\A\). Therefore, dD is simpUfied as 

. . N 

iapp^l-—Y. LUTl]Kapp{n)\). (4) 

n=l 

Note that due to the averaging over all bit decisions, this 
approximation becomes more accurate as the iterative decod- 
ing converges to the reliable decision. When the decoding 
delivers reliable results, Kapp{n) values are generally high. 
Consequently, lapp — > 1 by (|4]l. In contrast, erroneous decod- 
ing will result in a lower lapp value due to smaller magnitudes 
of LLRs. For illustration purpose, three typical examples are 
shown in FiglT] - |2] They are based on decoding a (7, 5) 
code using random interleavers of length 900. Coded data are 
binary phase shift keying (BPSK) modulated and transmitted 
over AWGN channel. Fig. [T] is obtained from unsuccessful 
decoding of a packet at Ejj/No = IdB. The inner two curves 
in Fig. |2]are from successful decoding of another packet also 
at Eb/No = IdB, while the outer two curves in Fig. |2] are 
from successful decoding of a packet at Ei,/No — 3dB. 
The evolution of lapp during the iterations is plotted: The 
vertical axis, denoted by I^pp, refers to the mutual information 
produced by the 1** constituent decoder. The horizontal axis, 
denoted by lapp, refers to that produced by the 2"'* constituent 
decoder. By sequential exchange of extrinsic information, each 
decoder accepts the mutual information produced by the other 
decoder as the input, and generates its own mutual information 
as the output. The two curves in each plot indicate how each 
decoder's output mutual information evolves according to the 
input mutual information it receives from the other decoder. 

III. e THRESHOLD AND TYPE-I STOPPING RULE 

Based on the characteristics shown above, we propose a mu- 
tual information aided (MIA) iteration stopping rule: decoding 
stops when 7^^^ is very close to 1, i.e., e is small enough. 
Q We denote this stopping rule as type-I mutual information 

'it is reasonable to apply the threshold on lapp also for 1st decoder, which 
will result in decoding stops 0.5 iterations earlier for some cases. 
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Fig. 1. Evolution of tlie approximated mutual information: Decoding 
convergence not reached (Ei,/No = IdB). 



DEC #1 

-e — DEC #2 



1 

0.9 
0, 
0.7 
0.6 
: 0.5 
0.4 
0.3 
0.2 
0.1 





DEC #1 

-e— DEC #2 



0.2 



0.4 



0.6 



0.8 



Fig. 2. Evolution of the approximated mutual information: Decoding conver- 
gence reached at low SNR (Eb /No = IdB) and high SNR (Eb /No = 3dB). 



aided (MIA-I) rule. For simplicity, a fixed threshold value on 
e (e.g., 10"^) may be applied. For flexible iteration stopping, 
more specifics about e are considered as follows. 

A. Estimation of Bit error rate and e threshold 

According to the definition of LLR, it is straightforward 
to derive the conditional bit error probability given the a 
posteriori LLR A as 



1 



1 + elA| ■ 
For large |A| it then follows that 



^'elA — 



1 



1 



4«ln(l 



(5) 



(6) 



For reasonably large |A| (e.g., for |A| > 2), we can verify that 
the difference from approximation is negligible. Therefore e in 



dUi could also be used for a rough estimation of decoding bit 
error rate (BER). Note that LLRs produced by early iterations 
are more accurate but less so in later iterations due to increased 
correlations with soft input to the decoder(s) ifTTI . For this 
reason, e approximates BER better in early iterations but not 
as well in later iterations (as it usually approaches zero with a 
large number of iterations). For effective iteration stopping, the 
threshold e may be selected to be around the value of expected 
BER. Consider a fixed threshold of e, it is intuitively clear that 
a lower threshold ensures lower BER but may require more 
iterations. This provides the following two options of selecting 
a threshold of e. 

Option A: When decoding throughput is of higher priority 
but BER is less important, a relatively high threshold of 
e can be applied. This may be appropriate for certain real 
time applications such as voice or video transmissions with 
certain quality of service (QoS) requirement. In some practical 
scenarios BER of lO^'^ to 10^"* can be acceptable. In those 
scenarios, a fixed threshold on e may be chosen from, e.g., 
10~^ to 10^'', for the desired BER. For notation purpose we 
denote MIA-I rule with fixed e threshold by MIA-I-A. 

Option B: When decoding reliability is of higher priority, an 
estimation of achievable BER can be used for e threshold. For 
known channel and coding scheme, BER can be measured 
in advance. If such a BER value is unknown a priori, a 
sufficiently low e may be used initially, and then adjusted 
accordingly once BER can be measured or estimated. This 
may apply to channels with slow changes. We denote MIA-I 
rule with such adaptive e thresholds by MIA-I-B. 

B. Simulations 

The effects of different thresholds are illustrated by simu- 
lations. The PCCC encoding scheme uses two identical (7,5) 
component encoders, with random interleaving of size 900. 
Puncturing of even (odd) indexed parity bits by first (second) 
encoder is applied to generate the rate 1/2 code. Coded 
data are modulated with BPSK and transmitted over AWGN 
channel. 

For MIA-I-A, we applied e = 10"^, e = 10"^, and 
e = 10^^ for different trade-offs between BER and number 
of iterations. For MIA-I-B, we adaptively set e according to 
Table I] to match the achievable BER. We compare these MIA 
stopping rules with the CE rule as well as the HDA ruleHH. 
The threshold for the CE criterion is lO^"*. All stopping rules 
are then compared with decoding using 6 iterations. 

We plot BER curves as well as the average number of 
iterations in Fig. [3] and Fig. |4] As a reference, BER as well 
as the average numbers of iterations assuming a 'genie' error 
detector are also plotted. Its number of iterations refers to 
either the lowest number of iterations for error free decoding, 
or the maximum number of 6 iterations if errors always exist. 

^Other stopping methods, e.g., SCR, SDR or IHDA, etc, are similar to or 
slightly worse than CE or HDA in perfonnance and/or average number of 
iterations as reported in I?], lH] and (9]. 

Also note that the proposed MIA rules can also compare lapp or € ratios 
after each half iteration with trivial modification, which could further reduce 
the average iteration number. 




TABLE I 

Threshold of e for different SNR 



(7,5) code with interleaver size 900, AWGN channel 
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Fig. 5. cross-entropy ratio and e ratio over iterations: rate 1/2 turbo code 
with memory length 2, interleaver size 900, over AWGN channel. 



iteration number as iter and consider extrinsic and a poste- 
riori LLRs output from second component decoder , CE is 
expressed as 16i 



Fig. 3. Performance by different e threshold: rate 1/2 turbo code with 
memory length 2, interleaver size 900, over AWGN channel. 



IV. e RATIO THRESHOLD AND TYPE-II STOPPING RULE 



CE{iter) 



1 



N 



N ^ 

n=l 



AA^*'='^(n)| 



(7) 



where AA**^'' = A^*'^'' - A^*'='"~i. 

The effectiveness of CE rule comes from a fact that the 
ratio of '^^g*^)''* typically shows an accelerated decreasing as 



To avoid estimation of BER for determination of lapp decoding converges. Decoding stops if ^cEii)'^ < ^^^^ 



threshold, we further consider the metric e. As shown by 
Fig. 12] decoding convergence accompanies the minimization 
of e. Interestingly, in average e{iter) shows a similar pattern 
as that of the CE criterion [6| that we can leverage. 

To illustrate we briefly review the CE rule below. Denote 



CEjiter) 
CE{1) 



(1) 

< 10'^. Usually choosing 10""^ saves slightly in 
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Fig. 4. Average number of iterations by different e threshold: rate 1/2 turbo 
code with memory length 2, interleaver size 900, over AWGN channel. 



iteration numbers but more often result in an early error floor, 
while 10^"' usually maintain performance better at the cost of 
slightly higher number of iterations. 

To show the pattern of '^'"^^^^^^^ in comparison to ^^^^fj^, 
we plot numerically such two ratios over iterations in Fig. |5] 
The figures are randomly generated by decoding 500 coded 
packets at Eb/No = 3 dB and measuring those two ratios 
by each packet after each iteration. In both figures one curve 
is plotted for each decoded packet. In average, ^^J*^^'''' drops 
faster. This implies the possibility of earlier stopping compared 
with the CE rule. 

Based on the observation, we express the second stop- 
ping criterion as < 10~^. We denote this stopping 
rule as type-II mutual information aided (MIA-II) rule. We 
further note that computation-wise, e{iter) requires N look- 
up table search, — 1 additions, and one multiplication, 
while CE{iter) requires N times look-up table search (for 
the exponential function), 2N — 1 additions, and 2N + 1 
multiplications. 

A. Simulations 

In this section we compare MIA-II and MIA-I-B with the 
CE rule as well as the HDA rule. Simulations are based 
on PCCC, with the (7, 5) component code with random 



TABLE II 

e Threshold values for different SNR 



(7,5) code with interleave! size 2048, AWGN channel 
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(7,5) code with interleaver size 2048, Rayleigh fading channel 
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Fig. 6. BER: rate 1/2 turbo code with memory length 2, interleaver size 
2048, over AWGN channel and fast Rayleigh fading channel. 

interleavers of size 2048. We compare different stopping rules 
for transmissions by BPSK over AWGN channel and fast 
Rayleigh fading channel. 

For MIA-I-B stopping rule, the threshold value is chosen 
as in Table |ll] The BER performance comparison is shown 
in Fig. |6l The average numbers of iterations by each rule are 
shown in Fig. [T] Compared with the CE rule or the HDA 
rule, the proposed MIA rules usually stops earlier 

V. Conclusions 

Using turbo code as example, we provided an approximate 
calculation of the mutual information between encoded bit 
and decoder's a posteriori LLR. The changing pattern of 
such metric effectively indicates decoding convergence after 
iterations. Two types of iteration stopping rules are proposed. 
The first type includes two options: MIA-I-A using a fixed 
threshold provides flexible trade-off between performance and 
complexity; MIA-I-B ensures the performance by an estima- 
tion of the achievable BER. The second type, MIA-II, adopts 
a more universal threshold with a strategy similar to that of 
the CE rule, but requires less computation. Simulations show 
that compared with CE and HDA stopping rules, both MIA- 
I-B and MIA-II achieve higher efficiency while maintaining 
achievable performance by a maximum number of iterations. 
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Fig. 7. Average number of iterations: rate 1/2 turbo code with memory 
length 2, interleaver size 2048, over AWGN channel and fast Rayleigh fading 
channel. 
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